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ABSTRACT 

Data from research studies seem to support the 
validity of teacher judgments of student achievement; however, there 
are unresolved issues, including the focus of the judgment, the 
external validity or credibility of the assessment, and the process 
of the assessment itself. To achieve maximum benefits from this form 
of assessment/teaching interface, teachers may need to be sensitized 
to the extent and importance of the assessment role in the teaching 
process. Intensive experience with basic principles of measurement 
and assessment instruments and other devices is needed. Effective 
teaching is based on an ability to assess accurately: assessment does 
and should inform the teaching process. It becomes more powerful as 
the assessment becomes integrated into teaching. The more a teacher 
uses intuitive judgement and direct observation of a student's 
literacy development, the more powerful the assessment becomes in the 
classroom. The key to the development of a cohesive and coherent set 
of assessment tasks all directed toward the same developmental aim is 
the existence of a criterion scale of development that can be used to 
link the teaching and learning process and the interpretation of 
assessment information. Linking assessment and reporting to teaching 
and learning is facilitated by the use of descriptive criterion 
scales which standardize the reporting terminology. The link between 
teaching, learning, and assessment, while having an undeniable 
internal validity, may at times need an external reference to 
establish external validity. (Two figures are included; 27 references 
are attached.) (RS) 
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In this paper we will examine the possibilities of integrating assessment with teaching and 
learning. In doing so we will examine the interactive process of question and answer in the 
classroom and how it closely parallels a more formal process of assessment and reporting. 

There is a growing body of jvidence that indicates how teacher judgements influence 
decisions in the interactive phase of Teaching (McNair, 1978, Hoge and Colardarci, 1989). 
The teachers main consideration during reading instruction appears to be reading 
achievement. Teachers pace whole class instruction on the basis of whether an identifiable 
group of students understand what is being presented (Clark and Peterson, 1986). 

Questions are used in the evaluation of pupil comprehension, learning, thinking, knowledge 
or task performance. (Colker, 1984). All this supports the idea that teacher decision-making, 
particularly in an interactive context, is influenced by judgements about student learning. 

The issue is whether the teacher judgement is accurate. The implications are important when 
judgements inform decisions regarding students for feedback, reporting to parents and other 
stakeholders (Elliott, Gresham, Freeman and McClosksy, 1988). Teacher judgements 
provide the primary data for most classroom decisions. It is widely assumed that teachers 
are generally poor judges of student attributes and that this is due to lack of perception, bias 
and error. "Directly or indirectly, the accuracy of teacher's assessments of student ability 
is often an issue in educational research. It is commonly argued that commercial tests 
provide teachers with valuable information about the abilities ana deficiencies of their 
students, from which it follows that teachers who rate their students without such information 
will be in error." (Egan and Archer, 1985, p.25) 

Hoge and Colardarci however, through meta analysis of a range of studies, have concluded 
that teachers differ in how accurately they judge their students* achievement. There was a 
generally high level of agreement between judgemental measures and standardised 
achievement test scores. A median correlation of 0.62 was found. Griffin (1990) replicated 
this level of ap.reement when teachers were provided with a descriptive criterion scale on 
which to have the judgement of achievement. 

Data from research studies seem to support the validity of teacher judgements of student 
achievement. Studies of both convergent and concurrent validity have been consistently 
higher than those reported for psychological tests, (Hoge and Colardarci, 1989), despite wide 
variations in methodology across studies. However there are still some unresolved issues. 
These incluUe the focus of the judgement, the external validity or credibility of the 
assessment, and the process of the assessment itself. 

The Focus: 

Judgemental assessments do not always make it clear as to what aspect of student 
performance is being assessed. The work of Griffin (1990) and of Farr and Fan (1991) offer 
some guide in that both provide descriptive scales as a frame of reference for the judgements, 
and illustrate how the judgement is closely allied to the teaching and learning process. In 
providing the descriptions of increasing proficiency, the materials circumvent the issue of the 
validity of standardised test data. Moreover, problems associated with global judgements 
(high, low, etc.) are avoided or at least contracted by reference to the descriptive scales. 
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Coladarci's (1986) criticism is avoided because much is disclosed about the teachers' specific 
knowledge of what the student has and has not mastered in some domain: In the case of the 
Griffin (1990) and Farr and Farr (1991) projects, the domains are Reading and Writing 
respectively. As yet, however, too little research has been done on convergent ana 
discriminant validity of judgements based on criterion scales. 

In order to achieve maximum benefits from this form of assessment/teaching interface 
however, teachers may need to be sensitised to the extent and importance of the assessment 
role in the teaching process (Hoge, 1983 and Hoge and Cudmore, 1986). Intensive 
experience with basic principles of measurement and assessment instruments and othei 
devices, including norm referenced tests, observational procedures, and judgemental scales, 
are needed. Many more judgement scales need to be developed in line with Glaser's (1981) 
criterion referenced interpretation. Using this experience and improved assessment 
technology, there is a need to enhance teachers' abilities at analysing and diagnosing learning 
in children. A great deal of importance is attached to the judgements of teachers in both the 
teaching process and in communications with stakeholders in the education process. 

The Nature of the Assessment: 

Griffin & Nix (1991) define Assessment and Reporting as..."tfze purposeful process of 
observing, interpreting, reporting and communicating information about student learning to 
relevant stakeholders" (p. 10). We can take each pan of this definition and examine it for 
its relevance to teaching and learning. Assessment and reporting, and teaching and learning, 
are both purposeful processes. Each has as its major purpose to improve, assist and 
encourage learning. They are processes that continue throughout the teaching and learning 
procedure in the classroom. Assessment and reporting is not an event. It is not, as is often 
believed, a single test at the end of term and a report that goes home to parents. Assessment 
and reporting is not a one-off thing. It h a continuous process that underpins most, if not 
all, human interaction. It is the process of gathering information through various forms of 
observation. Each time two people interact, exchange information, cooperate, negotiate, or 
work together in any way, assessment takes place. 

Information is gathered and interpreted in order to make sense of the observations that are 
being made. Several interpretive frar .'works are used. We interpret what we see in terms 
of what we expect to see. That is a norm referenced assessment. We interpret what we see 
in terms of the actual behaviour and our knowledge of that kind of behaviour. That is 
criterion referenced interpretation. Alternatively the person can interpret their own behaviour 
for us. This is Ipsative referenced interpretation. We need to make sense of information and 
observations through all three frames of reference and to record the meaningful information. 
Interpretations are recorded in a notebook, z. folio, or sometimes it is recorded as an object, 
a product of a students work - a sample. Sometimes it is recorded in the observers memory. 
The last part of assessment and reporting definition involves the communication that takes 
place. If it is to parent!., it may well be a via parent/teacher interview, a formal term report, 
or a detailed letter frort the school. If it is to the student, k may be simple feedback about 
performance on a classroom task. If it is to the school district, it may be the aggregate 
s ,ores of the class on a test, or description of the kinds of goals and targets set for the 
students. 
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However, assessment and reporting is not a strictly linear process from purpose through 
observation, interpretation, recording and communicating to a particular audience. 
Collecting assessment information never stops. A teacher's intuition, judgement and 
observation skills are constantly being used. It is interesting to note that standardised tests 
were originally introduced to compensate for poor teacher judgement. We now know a lot 
more about learning, and about reading in particular. We now understand that there are few 
standard sed tests that can reflect the learning process as well as the development of reading 
skills, in addition teachers are more skilled in observing and interpreting children's reading 
behaviour, and a new form of standardised assessment is emerging th&; capitalises on 
teachers expert judgement. We can now recognise and use the fact that teachers are 
continuously and intuitively n«aking judgements about student learning . 

Inside the classroom the teacher is continuously reporting by feedback and what Black (1987) 
calls feed forward. Let's take an example of self assessment and include a report. In the 
figure below, the passage "Wlten h a Word Not a Word" (Hopkins, 1984) is reproduced. 
Take a few minutes to read the passage. Monitor what you are doing when reading on, 
reading back, and note where yot, use syntactic, semantic, contextual, and giaphophonic 
cues. Where do you risk being wrong, omitting something all together? What do you use 
to make meaning? Is your guess consistent with your knowledge of words? Do you learn 
anything? 



Placv Hopkins Figure Here. 



When you have "read" the passage turn to your neighbour and ISSQH on your experience. 
While reading, you were assessing or gathering information about your own reading 
strategies (metacognition). You recorded these (mentally), interpreted them in terms of your 
own knowledge of reading (criterion referenced), and communicated your interpretation to 
a peer. This is assessment and reporting. 

In teaching we observe what students do, make sense of it, keep records, make decisions, 
and act on it. The process is fundamentally the same. Effective teaching is based on an 
ability to assess accurately. Assessment does and should inform the teaching process. It 
becomes more powerful as the assessment becomes integrated into teaching. The more a 
teacher uses intuitive judgement and direct observation the more powerful the assessment 
becomes in the classroom. In fact the more we investigate the assessment and reporting 
process the more it sounds like teaching and learning. Goals are set, negotiated, explored, 
and a purpose is established. Throughout the deliver) of instruction, teachers and students 
are continuously observing. Teacher observing student, student observing teacher, student 
observing student. In this interactive process, information is exchanged, interpreted, and 
recorded. The teacher gathers information about the students, the students gather information 
about the teacher, about each other, and about the subject matter. They use their existing 
knowledge base to make sense of their observations and integrate it, thus expanding that 
knowledge base. This becomes a recording mechanism. On the basis of formal and 
informal records and interpretations, the teacher makes decisions about what steps to take 
next. The teacher continuously provides feedback. 
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Take the simple question and answer process. There is always a purpose. The teacher asks 
a question, receives an answer, interprets that answer in terms of an expected response, and 
ie terms of the knowledge, skills and attitudes that are expected to be developed. A record 
is made instantly, at least in the mind of the teacher, and some feedback is given to the 
student. More th&n likely some feedback may come from the student to the teacher. Hence, 
in simple classroom interaction the formal process of assessment and reporting is replicated. 

Validating the Process: 

What we want to do now is to examine how it is best done, take a look at some examples 
in the language curriculum, and explore possibilities of how these examples might be linked 
together. 

In both teaching and learning, and assessment and reporting, we need to understand what is 
expected to occur. In some forms of assessment this is often hidden or taken for granted in 
a series of tests, exercises or work samples. But, if these tests, work samples, and exercises, 
w re collected together to form a cohesive and coherent set of assessment tasks all directed 
toward the same developmental aim, we have a powerful eliciting and interpretative 
framework for a teacher to make sense of observations in the classroom. The key to this 
interpretative framework is the existence of a criterion scale of development that can be used 
to link the teaching and learning process and the interpretation of the assessment information. 
Linking assessment and reporting to teaching and learning is facilitated by the use of 
descriptive criterion scales which standardise the reporting terminology. They free the 
teacher to assess and teach in a manner most suitable to the particular class while reporting 
within a common frame of reference. There are several examples of such a model. The 
Literacy Profiles exemplified in Figure 2, chart the development of reading and writing and 
provide advice to teachers about the kinds of classroom exercises that can be used to elicit 
the behaviours included in the scales. In a remarkably similar scale, the United Kingdom 
National Curriculum, targets and levels for reading, writing, speaking, and listening describe 
increasingly proficient literacy behaviour. 



Place Figure 2 Here. 

Pages from Teacher's handbook. 



Another set of scales has been developed by Griffin & Forwood, (1991). In the United 
States, the NAEP scales, although derived from test items, provide the same kind of 
interpretative framework. The College Board (1986) developed the testing system, the 
Degrees of Reading power, which provides a different kind of descriptive scale. This scale 
describes reading development in terms of the kinds of reading material that can be handled 
rather than in terms of the reading skills being developed. Farr and Fair (1991) have 
produced a series of writing criterion scales, tied to set reading passages. 

These examples all have in common an underlying criterion scale. The Degrees of Reading 
Power scale hs* been included because, while this is a set of cloze passage reading test, its 
scores are interpreted in terms of the kinds of reading material that a student can be expected 
to demonstrate or master. The scales make it easy for a teacher to monitor a student's 
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progress almost by direct observation of the kinds of reading material, the kinds of reading 
behaviour, attitudes, and reading activity within the classroom. 

The teacher is guided in each of these schemes as to the kinds of activities that are most 
likely to illicit reading or writing behaviour described in the scales. Once the teacher begins 
to develop reading activities, which will develop and illicit the indicative behaviours included 
in the scale, assessment and teaching and learning become indistinguishable. 

This is a very different aspect to the washback effect of testing. It is not teaching to a test; 
it is teaching and learning deliberately designed to illicit indicative behaviours which indicate 
developing proficiency in language skills. It is only when the teacher recognises these 
behaviours that they can conclude that the student is developing and moving forward on the 
descriptive criterion scales. Classroom exercises to encourage learning to occur, and direct 
observation is used to determine whether the indicative behaviours are present. The 
observations may then be directly interpreted in terms of the location of those descriptors on 
the scales. This is criterion referenced interpretation of assessment information. It is 
assessment of the direct behaviour being demonstrated and the relationship of that behaviour 
to other indicative behaviours, which together, form a cohesive description of developing 
proficiency. 

The criterion scales can legitimately drive the curriculum by enabling the teacher to set 
realistic goals for the student and negotiate regarding the kind of classroom activities to be 
undertaken in order to progress along the scale. The more we begin to work with descriptive 
criterion scales, the mo-.-e difficult it becomes to separate teaching and learning from 
assessment. The process of observing, interpreting, recording, and communicating becomes 
teaching, and teachers can base their work on a cohesive developmental progression for the 
students. It allows the teacher to make explicit the intuitive process of assessment in 
interaction with students. The teacher/student relationship can be articulated through the 
activity in the classroom xnd the relationship of those activities to the developmental 
proficiency in the language skills. It gives the assessment, and teaching and learning, 
internal validity within the closed system of the classroom. The assessment being so solidly 
linked to a teaching learning process, and in fact, indistinguishable from it. This means that 
the assessment data has unarguable internal validity. 

However, there is a note of caution. The internal validity of the classroom process must be 
matched by its credibility to external audiences. The assessment and the reporting process 
must have external validity. Outside observers also have expectations and may also have an 
understanding of the kinds of behaviours that link and act together to form a cohesive and 
coherent definition of developing proficiency. External observers may also have a vested 
interest in the way in which progress in learning is demonstrated. To maintain credibility, 
teachers' assessments must also have external validity. 

It is important that the assessment information and its interpretation should be able to be 
generalised to audiences, other than, but including the student, teacher, and parent. The 
communication needs to be in a form that makes it understandable, accurate and acceptable. 
It is an unfortunate fact that the teachers judgement is still not always accepted as accurate. 
As a consequence, the link between teaching, and learning, and assessment, while having 
undeniable internal validity, may at times need an external reference in order to establish 
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external validity. The external reference is sometimes in a form of standard assessment tasks 
that are acceptable to the teacher and the student, and can be part of the teaching assessment 
and learning process. They should provide a powerful form of communication to outside 
audiences, and establish the exter lal \ alidity of the assessment. The use of descriptive scales 
can become a method of bridging the gap from the internal validity of the classroom intuitive 
and judgement based assessment, to the externa) need for objectively based information that 
is considered to be credible and valid. 
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WRITING BAND B 



In a nutshell ... 

These students are hurtling about hand writing. 
They know what letters and words are and talk 
about these terms when they are writing. 




CONTEXTS FOR OBSERVING BEHAVIOURS 

iff students during WRITING 4 SHARED P£ADING session* are time* when 

SESSIONS allow* the hue her to continue to monitor students are able to share their developing knowledge of 

increasing ntnt.td and prelVieiwe of hand /nd writing the vocabulary of print, and to question tho&e elements 

implement Developing under standi ngs about the which are engaging I heir attention I tfiserv.mt teacher* will 

con venttuii^ of punt wnl he apparent, and tor each increase their knowledge about student* during shared 

>.-dent, these should t»e noted, with date*, tor records reading 

0 The WIUTINC; CONFERENCE * an opportunity 
for students to talk with teachers about their writing This 
may take the form of 'reading' what has been written, or 
disnjsMOf; the writing jnd/m dtawtng Such times aie 
nch ..nun t*>t>i mlt .rm.it i* m and reveal rm*« li about students 
understandings *»t thee tiiiveittuitwand vocabulary of pnnl 
and about the ttte.tnmg thry ate making in their wnting 

❖ 1*1 om SAM PI IS m the WRITING FO| IO. te* hers 
will be able to identify each student's development m the 
ir e of writing convent ions and the range of ideas and 
vocabulary in use The WRITING FOLIO itself may be 
used fot students to develop ideas for wnltng, as well as a 
list of the language a inventions the y are employing This 
enables the teacher n« tin ate uuiikly students who need 
Assistance %vith ideas toi witting, and also to celebrate 
achievemettU witti ttuii students 
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SCHOOL 



NAME 



LITERACY PROFILE 

..CLASS 



TERM 



CONTEXT OF 
OBSERVATION 



YEAR: 19* 
COMMENTS 



WRITING BAND A O 

What the writer doe*. L J - — 

Uses writing implement to make marks on paper. 
Explains the meaning of marks < a word, sentence, writing, letter). 
Copies "words* from signs in immediate environment 
"Reads", understands and explains own "writing". 

What the writing ahowa [> ' — 

Understanding of the difference between picture and print. 
Use of some recognisable symbol* in writing. 

Use of Writing □ 

Comments cm signs and other symbols in immediate environment 
Usts a mixture of drawings and "writing" to convey and support an 
idea 



B 



□ 



WRITING BAND 
What the writer does □ 

Reproduces words from signs and other sources in 

immediate environment. 
Holds pencil/pen using satisfactory grip. 
Uses preferred hand consistently for writing. 

What the writing shows D — 

Use of vocabulary of print — letters, words, question marks. 
Use of tetters of the alphabet and other conventional 

symbols. 

Use of letters in groups to form words. 
Placing spaces between groups of Tetters". 
Knowledge that writirg moves from left to right in lines 

from the top to the fc ottom of the page. 
Use of Writing \) 
Writes own name 



wajTTNC SAND C [~] 

WWt dw wnUi doc* ( 1 

C otnmcrKW wrutt\g w a howl &*M*Urv:e 

tit* « prnonfttttcrf hftnrfwntmg *tyle chji mert* nv*! t^ndwftUn^ rx\**i% 
ChM kft wntten wofk by renting it »U**d 
Sound* out wurd\ iiMndki »f 

Wh*< tf* *»riti«t *Ho*o |>- 

Ltftbte wfiimg with ncugmuMt wot da 
Word* put together tf» a »*fMcoce form* 

WcvJ* written «a*fof k«J ok4 « to awfce ■ arntrnc r th*t c i n be ft** 
Upper And \aw*t cue letter* **Kd convwiwoittr 
Written vntfncd that <**r> be wndcmaod by mi Mtutt 

L'm ai Wrttinf Q--. - . 

Sent mora convey u mi y on ope kiptc 
l>tr* "I" ta wnfttn| 

Wrrtr* sbouk fcettfip, judgement uf direct cpcncrxc 

C Fv*t<% cterjctert rom e*pencace *nd wvntdu* rrmnxunrrti 

trrttfcR AssessMtasns 



t l°rofttr» ftarord 



Mtnistry ot biw«ltorc VtrtxU 
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